Computer-implemented system and method for providing performance alerts in an online learning environment

ABSTRACT

A computer-implemented system and method for providing performance alerts in an online learning environment is provided. Education-related information for a user is monitored within an online learning environment and an assignment completed by the user is received. A score of the assignment is determined to be below passing level. Other assignments that are the same as the received assignment and are associated with below passing scores of the user are identified. A number of the received assignment and the same assignments is determined as common assignments. A predetermined threshold is applied to the number of common assignments and when the number of common assignments exceeds the predetermined threshold, access of the user to the online learning environment is blocked. An alert providing notice of the user&#39;s blocked access is delivered.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is a continuation of, U.S. Pat. No. 10,078,874,issued Sep. 18, 2018, which is a continuation of U.S. Pat. No.8,775,515, issued Jul. 8, 2014, which is a continuation of U.S. Pat. No.7,912,900, issued Mar. 22, 2011, the priority dates of which are claimedand the disclosures of which are incorporated by reference.

FIELD

The invention relates in general to online education, and in particular,to a system and method for providing performance alerts in an onlinelearning environment.

BACKGROUND

Education is important in our society. Each year, significant time andmoney are spent on improving the quality of education, increasing scoreson standardized tests, and stimulating students' interests in learning.Today's students come from a wide variety of backgrounds, socialeconomic levels, and ethnicities. Traditional teaching methods haveshown to be ineffective for all types of students, and alternativelearning methods provide some students with an education better tailoredto their specific learning needs. Alternative learning is oftenbeneficial because students are presented with educational materialsthrough non-traditional methods. For example, an alternative learningprogram can be administered to honors students or those students thathave fallen behind in class.

With the increase in use of the Internet, online education offers apopular alternative learning method. Online education allows eachstudent to work at his own pace and to access learning materialssuitable to his learning style. Students can participate from atraditional school setting or from home.

Generally, online education includes an electronic curriculum of coursesthat are offered as a stand-alone education or as a supplement totraditional education. Each course provides online lessons, studymaterials, and assignments, for which the student is responsible.Completed materials can be electronically graded or sent to a liveteacher. Should questions arise, the teacher and students cancommunicate via the Internet. In some settings, no face-to-faceinteraction ever occurs.

The lack of face-to-face interaction complicates obtaining the teacher'sattention. In traditional classrooms, students can simply raise theirhand or speak directly to the teacher. In contrast, an online teacher isnot readily accessible to her students. If an online teacher is notlogged on to her computer, she will be unable to determine whether aparticular student needs help or has a question.

Attempts have been made to facilitate communication between educators,students, and parents in an online learning environment. U.S. PatentPublication No. 2006/0127870, to Fields et al. (“Fields”), disclosesnotifying an educator, student, or parent that an event requiring aconference has occurred. For instance, if a student's grade falls belowa threshold, an alert is sent to schedule an off-line conference. Thealert is automatically generated by applying a fixed criteria notadjusted to student or teacher considerations, or on extensive factors.Moreover, Fields fails to provide recipient alert interpretation andtemporal alert expiration.

SUMMARY

Conventional online learning environments often lack directcommunication between individuals. The lack of communication can hinderthe effectiveness of the online learning environment. Education-relatedalerts and education support groups provide near real time communicationof event occurrences between individuals associated with the onlinelearning environment.

An embodiment provides a computer-implemented system and method forproviding performance alerts in an online learning environment.Education-related information for a user is monitored within an onlinelearning environment and an assignment completed by the user isreceived. A score of the assignment is determined to be below passinglevel. Other assignments that are the same as the received assignmentand are associated with below passing scores of the user are identified.A number of the received assignment and the same assignments isdetermined as common assignments. A predetermined threshold is appliedto the number of common assignments and when the number of commonassignments exceeds the predetermined threshold, access of the user tothe online learning environment is blocked. An alert providing notice ofthe user's blocked access is delivered.

Still other embodiments will become readily apparent to those skilled inthe art from the following detailed description, wherein are describedembodiments by way of illustrating the best mode contemplated. As willbe realized, other and different embodiments are possible and theirseveral details are capable of modifications in various obviousrespects, all without departing from the spirit and the scope.Accordingly, the drawings and detailed description are to be regarded asillustrative in nature and not as restrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram showing an online learningenvironment.

FIG. 2 is a functional block diagram showing an online learningenvironment with an education support group.

FIG. 3 is a functional block diagram showing a system for providingeducation-related alerts in the online learning environment of FIG. 1.

FIG. 4 is a data flow diagram showing categories of education-relatedinformation.

FIG. 5 is a process flow diagram showing a method for providingeducation-related alerts in the online learning environment of FIG. 1.

FIG. 6 is a data flow diagram showing processes for identifying anevent.

FIG. 7 is a process flow diagram showing a process for generating analert.

FIG. 8 is a process flow diagram showing a process for delivering analert.

FIG. 9 is a process flow diagram showing a process for interpreting abenchmark.

FIG. 10 is a screen shot showing, by way of example, a Website forentering user preferences.

FIG. 11 is a screen shot showing, by way of example, an alert displayedon a Web page.

FIG. 12 is a screen shot showing, by way of example, an alert receivedvia email.

FIG. 13 is a block diagram showing, by way of example, an alert receivedvia a text message.

FIG. 14 is a screen shot showing, by way of example, an alert withresponse action elements.

DETAILED DESCRIPTION

Increased use of the Internet has led to more options in educationthrough online learning. Online learning provides an alternative forin-classroom learning, which is not always suited to different learningstyles and levels. FIG. 1 is a functional block diagram showing anonline learning environment. A registered student can access onlinelearning material 18 and enter user preferences 17 through a computer 11that is connected to a server 14 via an internetwork 16, such as theInternet. The online learning material 18 and user preferences 17 arestored in a database 15 that is coupled to the server 14. The onlinelearning material 18 can include homework assignments, lesson contentand plans, online tests, progress reports, and other material. Ateacher, often remotely located from the student, is able to post lessonplans, track students' progress, and obtain posted material via acomputer 12 that is also connected to the server 14 via the internetwork16. The teachers and students can communicate with a service center 13for an online education provider that provides technical support andmanages the online curriculum. The service center 13 is remotelyconnected to the internetwork 16.

Conventional online learning environments do not require the student andthe teacher to be logged on to their respective computers at the sametime. Instead, communication can occur through email, discussion boards,or instant messaging. Since students can access the online learningenvironment at any time, the online teacher is often unavailable forimmediate questions, unlike a traditional classroom where a student canraise his hand or speak directly to the teacher. In addition, otherpeople having an interest in a student's progress, such as parents,tutors, coaches, and counselors, often do not have access to thestudent's status or grades. Thus, these individuals often rely on thestudents for updates on progress. The lack of direct one-to-onecommunication, as well as absence of face-to-face interactions betweenteachers, students, and other individuals, can hinder student progressand decrease overall effectiveness of online education as an alternativefor traditional teaching methods.

An online learning environment differs from a traditional classroom inwhich a class of students receive in-person instruction from a liveteacher, who provides educational lessons, testing, grading, feedback,and in-classroom activities. The teacher is generally available fordiscussion with or to answer questions from the students when class isin session, before or after class, or during office hours. Further,interested individuals, such as parents or guardians, can monitorstudent progress and communicate with the teacher through report cards,parent-teacher conferences, and telephone calls.

The ease of initiating communication in a traditional classroom settingcan also be provided by the online learning environment througheducation-related alerts and education support groups. The alerts andgroups help to increase the effectiveness of the online learningenvironment by providing surrogate channels of communication. FIG. 2 isa functional block diagram 20 showing an online learning environmentwith an education support group. In the online learning environment,individuals 21, 24, 25-27, 28 can be associated with the online learningenvironment directly or through another individual, such as a particularstudent enrolled. The individuals can include a teacher 21, counselor24, student 25-27, parent 28, coach 31, or members of the onlinelearning staff (not shown). Other types of individuals are possible. Theeducation support group includes automatic groupings orparticipant-selected groups of the individuals. Near-simultaneouscommunication of an event occurrence in the online learning environmentis provided to the individuals in the education support group.

In the online learning environment, online teachers 21 are oftenremotely located from their students 25-27. The students 25-27 can beremotely located from one another or can be located in the same room,such as a traditional classroom. Each student 25-27 enrolled in theonline learning environment participates in an online curriculumprovided through an Internet accessible computer 32-34. The onlinecurriculum includes online courses offered by the online learningenvironment and the material associated with the online courses.

Once logged on, a student 25-27 can request particular learning material35 for an online course, such as a lesson, assignment, or test fordisplay on the computer 32-34. The requested learning material 35 isstored on a database 23, which is coupled to a server 22. The server 22obtains the requested learning material 35 from the database 23 forserving to the student's computer 32-34.

The online learning material 35 is part of the education-relatedinformation 31 maintained by the online environment. The database 23stores the education-related information 31, which can be served to thestudent's computer 32-34 via the server 22. The education-relatedinformation 31 can include all information in the online learningenvironment that is related to online educational activities, includingthe planning, offering, and processing of the online curriculum, as wellas a functioning of a system that provides the online learningenvironment. More specifically, the education-related information 31 caninclude performance, homework, communication, enrollment, system, andscoring data. However, other types of education-related information arepossible. Education-related information is discussed below with furtherreference to FIG. 4.

An online education provider, including an online learning staff,supervises the online learning environment and maintains theeducation-related information in the database by generating rules forproviding an effective online curriculum. The rules can includeprocesses for assigning passwords, offering courses, selectingcoursework, identifying cheating, measuring student progress againstestablished objectives and benchmarks, establishing alternativeeducational paths through the curriculum, offering additionaleducationally relevant material to support student learning,administering tests and other coursework, and providing status reports.Other types of rules are possible. The online learning staff can accessthe education-related information associated with individuals, includingall parties having a relationship to the online learning environment,such as the teacher, student, parents, mentors, counselors,administrators, and members of the online learning staff. The members ofthe online learning staff can observe behaviors exhibited by theindividuals, as well as patterns of performance based on theeducation-related information. Reports and statistics can be generatedto show the behaviors, performance, and results of the individuals inthe online learning environment, which are then provided with an alert.

The education-related information can also be used to identify events.An event is related to online educational activities, such as performingan action by an individual in the online learning environment,determining an individual's accessibility to use the online learningenvironment, and determining an online learning status for anindividual. Other types of online educational activities are possible.The event can be identified by a change to the education-relatedinformation 31, a request for an event, satisfaction of a threshold,change with respect to a baseline or determined by an expert system. Achange includes an addition, deletion, or modification of theeducation-related information. For example, a change to the performancedata could be an addition of a student score to an electronic gradebook. The student received a score of 74 percent on a test, which isadded to the electronic grade book by the online learning environment orassigned by a teacher. The event is the entry of the score the studentreceived on the test, which is related to a student action of taking theexam and the online learning environment or teacher action of gradingthe exam. In a separate example, a credit recovery organization haspreregistered for a particular number of students, which represents anenrollment threshold. Once the enrollment threshold is satisfied, anevent is identified and an alert is sent. The event is the satisfactionof the enrollment threshold by the credit recovery organization.

During event identification, a benchmark is determined. A benchmarkincludes information related to the event, such as a score, report,notification, status, message, or quantity. The benchmark informationcan be determined based on the education-related information that waschanged, collected from, or determined during the occurrence of theevent. Other types of benchmark information are possible. The benchmarkrepresents a starting point for interpretation of the event by arecipient of the alert. Event identification is described below withfurther reference to FIG. 6.

After identification of an event, an alert is generated and transmittedfor providing real time notification. An alert includes a templatehaving filled or fillable text for information regarding the event, suchas the benchmark. Recipient information and response action elements canalso be included in the alert. Returning to the previous thresholdexample, once the enrollment threshold has been satisfied, an event isdetected and an alert is transmitted. Additional information can beincluded in the alert such as charts, graphs, statistics, and previousbenchmark information that is related to the event. As described above,the online learning staff or online learning environment can generatethe additional information. Alternatively, an individual can prepare theadditional information; however, the individual's access to theeducation-related information can be restricted. Alert generation isdiscussed below with further reference to FIG. 7.

Recipients of an alert can include an individual or an education supportgroup, which includes multiple individuals with interest in theeducation-related information or in a particular student enrolled in theonline learning environment. The individuals in a an education supportgroup can include a teacher 21, student 25-27, parent 28, administrator(not shown), coach 31, counselor 24, paraprofessional (not shown), tutor(not shown), or a member of an online learning staff (not shown). Othertypes of interested individuals are possible. For example, an educationsupport group including a parent, coach, and counselor is generated fora particular student. The parent and counselor are concerned with thestudent's past scores and would like to closely monitor new scoresreceived. In addition, the student is involved in basketball; however,he is not allowed to play if his grades fall below a predeterminedthreshold. The coach receives alerts with the new scores to ensure thatthe student is eligible to participate in basketball games. Alertdelivery is discussed below with reference to FIG. 8. Upon receipt, eachrecipient can analyze the alert to determine a significance of theevent. Alert interpretation is discussed below with further reference toFIG. 9.

The online learning environment can be modeled after a traditionalclassroom. For instance, the online learning environment can include ateacher and class that meet virtually through the online learningsystem, as described in FIG. 1. The online learning environment can beimplemented in a variety of organizations, including a traditionalclassroom or school, an after school program, a credit recovery program,an honors program, a seminar, and for individual students. Additionally,the online learning environment can be modified to include multipleteachers or no teachers. Other types of online learning environments andmodifications are possible.

Alert System

The alerts increase the effectiveness of an online learning environmentby prompting and facilitating communication between teachers, parents,students, administrators, and members of the online learning staff, aswell as others. FIG. 3 is a functional block diagram showing a system 40for providing an education-related alert in the online learningenvironment 41 of FIG. 1. The online learning environment manages andtracks education-related information to identify interesting events,which are transmitted to one or more recipients via alerts. The alertsprovide a near real time communication between teachers, students,parents, administrators, and members of the online learning staff. Thealerts can cover a broad range of topics that are important to providinga successful online learning environment.

A watcher 42 monitors the online learning environment 41 by trackingeducation-related information. The watcher 42 is a standalone programthat passively receives data from the online learning environment 41,such as the education-related information, including performance,homework, communication, enrollment, system, and scoring data. Othertypes of education-related information are also possible, including datarelated to a specific online learning environment, which is furtherdiscussed below with reference to FIG. 4. The watcher 42 identifiesevents based on changes in the education-related information, a requestfor an event, satisfaction of a threshold, a change with respect to abaseline, and as determined by an expert system. Identifying an event isdiscussed below with further reference to FIG. 6. A benchmark is thendetermined for the event.

Once identified, events are sent to a queue 43, which is a passivemodule that holds the events until obtained by an alert processor 44.The events can be ordered in the queue 43 by date or time of receipt, byimportance, or by subject matter of the event. Other methods fororganizing and holding events in the queue 43 are possible.

Periodically or on demand, the alert processor 44 selects an event fromthe queue 43 and generates an alert. To generate the alert, the alertprocessor 44 can select a template based on a type of the event, thebenchmark, the subject matter of the event, or other factors. Next, thealert processor 44 determines to whom the alert will be sent. Selectingrecipients for an alert is discussed below with further reference toFIG. 7. The alert processor 44 also determines a delivery mechanism forthe alert, which can include a Web site 47, email 45, SMS text messaging46, and instant messenger (IM) (not shown). Delivery of the alert isdiscussed below with further reference to FIG. 8.

Education-Related Information

A wide range of information is generated during operation of the onlinelearning environment and various pieces of the information may be ofinterest to the individuals. FIG. 4 is a data flow diagram 50 showingcategories of education-related information 51. These categories broadlydefine topics of possible interest to the individuals, includingteachers, students, parents, guardians, mentors, administrators, andmembers of the online learning staff, as well as other individuals. Theeducation-related information 51 can include performance 52, homework53, communication 54, enrollment 55, system 56, and scoring data 57.However, other types of data are possible, including data that isspecific to a particular online learning environment. Each category willnow be discussed in detail.

The performance data 52 can include grade book information for astudent, such as test scores, absences, and student identificationnumbers. Homework data 53 can include whether a student turned in aparticular assignment and how well the student performed on theassignment. Performance 52 and homework 53 data is tracked to allowindividuals, including students, parents, teachers, and administratorswith an up-to-date status of student progress. Each alert can include areport of student activity or inactivity. For example, a student skipsschool and the teacher enters an unexcused absence into an electronicgrade book. The entry of the absence is identified as an event and analert is generated. The alert is sent to the parents to providenotification that the student was not present in class on that day.Other types of homework and performance data are possible.

Communication data 54 involves initiating a communication session with ateacher, student, parent, administrator, an online learning staffmember, or other interested individual or member of an education supportgroup. For instance, communication data 54 can include a question postedon a discussion board or selecting a help button on a Web pageassociated with the online learning environment. The action of enteringthe posted question or selecting the help button is identified as anevent for which an alert is generated. Scoring data 57 can includeinformation that indicates cheating, which is determined based on anumber of factors, such as time, score, and student identity. Forexample, a change in the scoring data can indicate suspected cheatingwhen a student takes an exam and receives a low score, and within ashort time period, re-takes the exam and receives a high score. An alertfor the suspected cheating event can be sent to the teacher oradministrator to provide notification that an unusual event has occurredthat may require action on behalf of the teacher or administrator. Othertypes of communication and scoring data are possible.

Enrollment data 55 can include information showing enrollment of a newstudent, withdraw of a registered student, or a hold status assigned toa registered student. Alerts regarding the enrollment data 55 can beprovided to an administrator and a member of the online learning staffwhen enrollment status changes to determine whether additionalsubscriptions to the online learning environment should be obtained.System data 56 can include notification of inoperability orinaccessibility of the online learning environment, such as rebootingthe online learning system or experiencing a power outage. Alertsregarding the system data 56 can be provided to the teachers,administrators, students, parents, and online learning staff to providenotification that the online learning environment may be inaccessiblefor use. Other types of enrollment and system data are possible.

Education-related information specific to an online learning environment58 can include mastery based learning, where each teaching segment orsection cumulatively builds on earlier learning. Consequently, masteryof prior material is key and a student who fails one section isgenerally held back until that section has been successfully mastered. Amastery based learning program requires a student to pass anend-of-section test prior to moving on to the next section ofeducational material. If a student repeatedly fails to pass the sectiontest, the student is blocked from accessing further materials and mustcontact a teacher or administrator, who may then unblock student accessor recommend further study or tutoring. The alert notifies the teacheror administrator that the student access has been blocked. The studentcan explain why he failed to pass within a predetermined number of timesor can ask for help to successfully pass the section.

Alert Process

Upon identification of an event, an alert is sent to one or morerecipients. The alert provides notification of the event occurrence innear real time. FIG. 5 is a process flow diagram 60 showing a method forproviding education-related alerts in the online learning environment ofFIG. 1. A watcher monitors education-related information 61 on aperiodic or continuous basis. If the watcher is set on a periodic basis,a member of the online learning staff can select predetermined timesintervals for monitoring the education-related information. The watchercan also monitor all of the education-related information or particularcategories of the education-relation information. A member of the onlinelearning staff can generate categories based on a type of theeducation-related information, such as performance, homework,communication, enrollment, system, and scoring data. Other types ofeducation-related information and categories are possible. The watcheridentifies an event 62 based on a change in the education-relatedinformation, a request for an event, a satisfaction of a threshold, achange in relation to a baseline, or as determined by an expert system.A benchmark is determined during the event identification, which isdiscussed below with further reference to FIG. 6.

After an event has been identified 62, the watcher determines whetherthe event necessitates an alert 63. Alerts provide near real timecommunication in an online learning environment. Currently, theindividuals in a conventional online learning environment experiencedifficulty in initiating communication based on the responsibility ofeach individual to proactively check and respond to emails and postingsto message boards. If one of the individuals fails to check the emailmessages or postings on a message board, communication between areceiving individual and a requesting individual cannot occur. Thealerts provide near real time notification of an event occurrence. If analert is not necessary, the watcher continues to monitor theeducation-related information 61. However, if the event requires analert, an alert is generated 64. Alert generation is discussed belowwith further reference to FIG. 7. Once generated, the alert is deliveredto one or more recipients 65 through one or more delivery mechanisms,including a Website, email, text messaging, or instant messaging.Delivery of the alert is further discussed below with reference to FIG.8.

Upon receipt of the alert, processing can optionally occur. Theprocessing can include recipient interpretation 66 of the alert based onthe benchmark for the event. The interpretation allows the recipients toform an opinion and come to a conclusion on their own regarding aparticular event and corresponding benchmark. Recipient interpretationis described below with further reference to FIG. 9.

Optionally, the alert can be associated with an expiration date. Oncethe expiration date is reached, the alert expires and is automaticallyremoved 67 from an email inbox, text messaging menu, instant messenger,or Website. The expiration date can include a time period, which beginsonce the alert is received or once the alert has been viewed by therecipient. Other types of expiration dates and time periods arepossible.

Identifying an Event

A variety of methods can be used to identify a particular event. FIG. 6is a process flow diagram showing processes for identifying an event.The event 71 can be identified by event rules, such as a change in theeducation-related information, a request for an event, a satisfaction ofa threshold, a change in relation to a baseline, or as determined by anexpert system. The change in the education-related information, whichcan include an addition 72, deletion 74, or modification 78 of theeducation-related information. For instance, an addition ofeducation-related information can include the addition of a studentscore, the return of completed homework, enrollment of a new student, ora new discussion board posting. In contrast, a deletion ofeducation-related information can include removal of an incorrect scoreor a student withdraw. Modification events can include changing astudent score or student status. Other types of addition, deletion, andmodification events are possible.

In addition, an event can be detected based on satisfaction of apre-determined threshold 73 by the event-related information. Forexample, each student must maintain a course grade of 75 percent;otherwise, the student is not allowed to participate in extracurricularactivities. Once a student falls below the 75 percent threshold, anevent is identified and an alert is sent to interested individuals, suchas a parent and coach. Other types of pre-determined thresholds arepossible. An event can also be identified based on a change in relationto a baseline. For example, a student receives a score of six out of tenanswers correct on an exam. Previously, the student received an averageof nine out of ten answers correct. The new test score is analyzedagainst the previous scores to determine whether the change issignificant to warrant the sending of an alert. Alternatively, thechange may not be significant and is thus, an indication that thestudent had a bad day. Other types of baseline changes are possible.

Further, an event can be determined by an expert system 75 or providedby an individual 76. An expert system can monitor the education-relatedinformation and detect an event based on one or more factors. Forinstance, cheating can be detected by considering multiple factors, suchas time, score, and student identity. A process is performed by theexpert system 75 to identify the cheating event. In addition, anindividual can request an event 76 by selecting a contact button orentering a notification into the online learning environment. Forexample, a parent may request communication with a teacher or the onlinelearning staff may enter a notification that the online learning systemwill reboot within one hour. Other types of expert system determinationsand individual provided events are possible.

A single event occurrence may trigger additional event occurrences togenerate a cascade of events 78. For example, upon enrollment of a newstudent, an event is identified and an alert is delivered to one or morerecipients. If the addition of the new student exceeds a number ofsubscriptions purchased, another alert is also generated. The newstudent alert and exceed subscription alert can be deliveredindividually or combined to form a single alert. Other methods forsending single or multiple alerts are possible.

During event identification, a benchmark is determined for inclusion inthe alert. In a first example, a teacher performs an online educationalactivity by assigning homework to the students in an online course. Thewatcher identifies the addition of the homework to the online learningenvironment and the education-related information as an event. Thehomework assignment is identified as the benchmark. Even though thealert was received, the student may not be required to complete thehomework if the student has already mastered the material or hasobtained the points required for the online class. Thus, the student candetermine whether he must complete the homework. In a second example, asite coordinator, such as an administrator, is responsible for thenumber of students enrolled in the online learning environment for aparticular organization. The organization can include a school, home,after school group, and credit recovery organization. The organizationis registered for a particular number of student enrollments, which istracked by the watcher. The organization performs an online educationalactivity by registering a student in the online learning environment.Once the number of students enrolled reaches or exceeds the number ofregistrations, an event is identified and the number of studentsenrolled is the benchmark for the event. Alternatively, the benchmarkcould include a number of students above the number of registrations,the names of the students, or a cost for the additional students. Thecredit recovery organization can interpret the benchmark to determinewhether they want to pay for the additional students or withdraw thestudents from registration. Benchmarks provide a starting point forrecipient interpretation of an event provided in an alert.

Once an event is identified, a watcher determines whether the eventrequires an alert to be sent. If required, an alert is generated.Otherwise, the watcher continues to monitor the education-relatedinformation.

Generating an Alert

Communication can be initiated between individuals associated with anonline learning environment using an alert to provide near real timenotification of an event. FIG. 7 is a process flow diagram showing aprocess 80 for generating an alert 81. An alert processor retrieves anevent with a corresponding benchmark from a queue 82 and selects atemplate 83 based on factors, such as a type of the event, thebenchmark, the subject matter of the event, or other factors. A templateis a predetermined format for an alert. A standard template can be usedfor all alerts or generated for a specific type of alert. Alternatively,the template can be customizable. Each template can include text,fillable fields, or blank text boxes that are filled by the alertprocessor. Also, an individual, such as a teacher, student, or parentcan manually fill the template. Other types of templates are possible.The alert processor then determines recipients of the alert 84. Therecipients can be selected from the individuals of an online learningenvironment, including teachers, students, parents, administrators,mentors, members of the online learning staff, or others, such ascounselors, paraprofessionals, and coaches. Alternatively, therecipients can include one or more members of an education supportgroup.

Education support groups help individuals to learn together. Theeducation support groups are formed from the individuals, which can beautomatically selected based on factors, including a type of event, asubject of the event, or based on a relationship with one or more of theindividuals, or as a combination of automatic and individual initiatedgroupings. For example, when a student is enrolled in the onlinelearning environment, an education support group, including a teacherand a parent is automatically generated. However, the student may alsowork closely with a counselor or mentor, who can also be added to theeducation support group. Other methods of grouping and selectingrecipients are possible.

After determining the recipients 84, the alert processor can determineone or more delivery mechanisms for the alert 85. The deliverymechanisms can include email, instant messaging, text messaging, anddisplay on a Website. Selecting a delivery mechanism is described belowwith further reference to FIG. 10. Optionally, response action elements86, such as action buttons, text recommendations, hyperlinks, and imagescan also be included in the alert. The response action elements caninclude interactive elements that allow recipients of an alert toperform a response action, such as obtaining or providing additionalinformation, initiating a communication, sending an additional alert,and restricting or allowing access to the online curriculum. Theresponse action elements can be selected by an individual, included withan alert template, or automatically selected based on a type of theevent. Other types of response action elements and methods for selectingthe response action elements are possible. The response actions andresponse action elements are discussed below with further reference toFIG. 14.

Once determined, the benchmark 82, template 83, recipients 84, deliverymechanisms 85, and response action elements 86 are compiled 87 to formthe alert. Each recipient of an alert can enter user preferences forreceiving particular types of alerts or for receiving the alerts througha particular delivery mechanism. Other types of user preferences arepossible. Upon alert generation and determination of the userpreferences, the alert is delivered.

Delivering an Alert

Each recipient can select to receive alerts via one or more deliverymethods. FIG. 8 is a process flow diagram 90 showing a process fordelivering an alert. An alert processor can access user preferences todetermine delivery mechanisms for sending the alert. Alternatively, thedelivery mechanisms can be automatically selected based on deliveryfactors, including a type of alert, designated recipients, or subjectmatter of the event. However, other types of delivery mechanisms, userpreferences, and delivery factors are possible.

The alert processor determines whether an alert should be sent to aWebsite 91 based on the user preferences or delivery factors. If not,the alert processor then determines whether the alert should be sent viaemail 94. However, if so, the alert is delivered 92 and displayed. Afterdelivery, the alert processor determines whether an additional deliverymechanism 93 is selected. If no other delivery mechanism has beenselected, the delivery alert process ends. Otherwise, if anotherdelivery mechanism is selected, the alert processor determines whetherthe alert should be sent via email 94 based on the user preferences anddelivery factors.

If email 94 has not been selected as a method of delivery, the alertprocessor then determines whether an alert should be sent via textmessaging 97. But, if selected, the alert is delivered via email 95.Once delivered, the alert processor determines whether another deliverymethod has been selected 96. If not, the delivery alert processterminates. However, if another delivery mechanism has been selected,the alert processor determines whether text messaging 97 has beenselected as a delivery mechanism based on the user preferences anddelivery factors. If not selected, the alert processor determineswhether another delivery mechanism 100 has been selected, whereas thealert is delivered 98 when text messaging has been selected. Afterdelivery, the alert processor determines whether another deliverymechanism 99 has been selected. If not, the delivery alert process ends.However, if another delivery mechanism 99 has been selected, the alertprocessor determines the type of delivery mechanism 100, and the alertis delivered using that delivery mechanism 101. The process ends afterthe alert has been delivered via all selected delivery mechanisms.

Upon receipt of the alert, each recipient can interpret a significanceof the event using a benchmark. The benchmark is determined from achange in the online learning environment, including a change ineducation-related information, from the information provided by anindividual to initiate an event, from threshold conclusions, fromchanges related to a baseline, or from conclusions by an expert system.

Interpreting a Benchmark

A benchmark provides a starting point for a receiving individual todetermine a significance of an event. FIG. 9 is a process flow diagram110 showing a process for interpreting a benchmark 111. An individualreceives an alert 112 with a benchmark for a particular event. Thereceiving individual can analyze the benchmark 113 by comparinginterpretation data, such as graphs, previous student information,thresholds, and averages with the benchmark. The interpretation data canbe included in the alert or accessed by the individual through theonline learning environment. After analyzing the benchmark, thereceiving individual can determine a significance of the event 114provided by the alert. The determination of significance can prompt theindividual to optionally perform a response action 115, includingobtaining or providing additional information, initiating acommunication, sending an additional alert, initiating a change to theeducation-related information, and restricting or allowing access to theonline curriculum and education-related information. The response alertcan include information directly entered by an individual or can begenerated based on a change in the education-related information.Response actions are discussed below with further reference to FIG. 14.

An example of benchmark interpretation involves a student who is takinga math test online. Once the student has completed the exam, an alert issent to the teacher to notify her that the exam has been completed. Uponreceipt of the alert, the teacher can then select to grade the exam atanytime. After grading the exam, the teacher enters the test score forthe student into the online learning environment. An alert is generatedwith a benchmark for the event, such as the score and delivered to thestudent. The alert notifies the student that he has received six out often answers correct. For this particular student, the score issignificant since on a previous math exam, the student received two outof ten answers correct. Thus, the student has improved considerablysince the previous math exam. However, another student who receives ascore of six out of ten answers correct may not conclude that the scoreis significant, such as a student who consistently receives the samescores.

In a further example, a member from the online learning staff requestsan alert having a benchmark that provides notification of a systemreboot to be sent to all administrators. The alert may be significant toone administrator who is generating reports via the online learningenvironment. However, the alert may be insignificant to anotheradministrator who is on vacation. As described, different conclusions ofsignificance can be formed from the same benchmark.

The alerts provide a near real time method of notifying recipients thatan event has occurred. The recipients are provided with a benchmarktransmitted in the alert to determine a significance of the event andwhether a response action should be taken. Each recipient can customizethe types of alerts received and the delivery mechanisms for receivingthe alerts.

Response Actions

Once a recipient has determined a significance of the alert, responseactions can be performed. The response actions can be performedindependently by the recipient or based on suggested response actionsprovided by the alert. The suggested response actions can be displayedto a user by response action elements, such as action buttons, textrecommendations, hyperlinks, and images, which are displayed in thealert message received. The text can provide a process or series ofsteps to perform in regards to a particular event, whereas, the actionbuttons can perform a particular action when selected by the recipient.The suggested response actions can be provided based on an expertsystem, related curriculum, and a related event. Other determinationsand provisions of the suggested response actions are possible. Theexpert system can generate the suggested responses using expertknowledge to determine one or more appropriate response actions for aparticular event. The expert knowledge is derived from a collectiveexpertise of education professionals familiar with the functioning ofthe online learning environment. The response actions allow therecipients to obtain or provide additional information, initiate acommunication, send an additional alert, initiate a change to theeducation-related information, and restrict or allow access to theonline curriculum by selecting a response action element. The responseactions and response action elements are discussed below with furtherreference to FIG. 14.

The recipient can select one or more suggested response actions based onan interpretation of the benchmark for an event. For example, an event,such as cheating or suspected cheating by a student can be detectedusing knowledge from the expert system. Upon detection, an alert is sentto an online teacher as notification of the cheating or suspectedcheating event. The alert includes two response buttons and responsetext. If selected, the first button allows the teacher to restrict thestudent's access to the online curriculum and the second button allowsthe teacher to immediately notify the student. The text recommendationincludes information from the expert system, which describes how thecheating event was determined. For instance, in this example, thestudent received a perfect score. However, the student took the sameexam yesterday and received a score of 20 percent. In addition, thestudent only took one minute to complete the exam for which he receiveda perfect score.

The text also includes recommended steps for the teacher to follow withregards to the cheating event. For instance, a first step recommendsthat the teacher suspend the student's access to the curriculum. Asecond step suggests that the teacher review all the student's scoresprior to initiating communication. A third step suggests discussing theevent with the student over the phone, instead of through email. Basedon the results of the third step, additional and alternative steps arealso provided.

Alternatively, the response actions can be performed automatically onbehalf of a user based on an event occurrence. For example, a student isenrolled in an online credit recovery program with the online learningenvironment. The student completes a chapter exam and receives a scoreof three out of ten. Upon identification of the student's low score, astudy plan is generated by the expert system and transmitted to thestudent. The study plan is provided as an additional learning tool andcan include a guide to mastering the material by outlining importantsubject matter, by providing a timeline for studying particular parts ofthe subject matter, and by providing additional information regardingthe subject matter. When performed automatically, a notification of theresponse action is provided to one or more individuals, including forinstance, one or more members of an education support group or thestudent himself.

The notification can include a message describing the reasons forperforming the automatic response action and the automatic responseitself. Returning to our previous example, a parent of the student mayreceive a notification identifying that their student received a lowscore and in turn, was provided with a study guide. Alternatively, theparent may also be provided with a notification having a copy of thestudy guide. In addition, the student can also receive a notification ofthe automated response action, such as when a hardcopy of the studyguide is mailed. The notification informs the student that he will bereceiving a study guide. Other types of notifications are possible.

Alert Screenshots

Recipients can enter user preferences for customizing the type of alertsreceived and the delivery methods used. FIG. 10 is a screen shot 120showing, by way of example, a Website 121 for entering user preferences.The Website includes a tab 122, which displays a title regarding thesubject matter for the Web page. A tool box 123 is displayed on the leftside of the Website, underneath the tab 122. The tool box 123 listseducation actions that can be performed by the individual, includingmaintaining a grade book, generating reports, setting up announcements,setting up an online classroom, sending a class roster, preparingstudent correlations, changing personal information, preparing a courseorientation, and running a system checkup. A user preference box 124 islocated below the tab, on the right side of the Website 121. In the userpreference box 124, an individual is able to select his preferences fortypes of alerts received and alert delivery methods. Other types of userpreferences and displays for entering the user preferences are possible.

The individual can select to receive alerts from any of the categoriesof education-related information, described above with reference to FIG.4, by clicking on a checkbox, selecting a category from a drop downmenu, or by typing in a particular category. The individual can alsoselect delivery mechanisms for receiving the alert. The deliverymechanisms include a display on a Website, an email, or text message.Other types of delivery mechanisms are possible, such as instantmessaging. The individual may be required to provide additionalinformation to ensure delivery via a selected method. For example, toreceive alerts on a cellular telephone, the individual can enter histelephone number 127 and service provider 128. After entering the userpreferences, the individual can select a submit button 129 to store thepreferences. Other methods for entering and selecting user preferencesare possible.

An alert is sent once the user preferences are set and an event isidentified. FIG. 11 is a screen shot 130 showing, by way of example, analert 132 displayed on a Website 131. Receipt of a new alert 132 can bedisplayed on the Website 131 as a pop up message, an icon, a highlightedmenu option, or by an alert button 135. The individual can click on thepop up message, icon, highlighted menu option, or alert button 135 toview the alert. Alternatively, the alert 132 itself can be displayed asa pop up message 132 when the individual is logged in to the onlinelearning environment. Each alert 132 can include a title 133 for acategory of education-related information and a benchmark 134 for theevent that triggered the alert. Other methods for displaying an alert ona Website and notifying an individual that an alert has been receivedare possible.

The alert can also be received through email. FIG. 12 is a screen shot140 showing, by way of example, an alert 142 received via email 141. Anindividual can access his email inbox 145 to determine whether an alert142 has been received. The alert 142 can be identified by a title 146 ofthe event. The title 146 can also be used as a heading for the subjectmatter 143 of the email 141. The individual can select the email 141 todisplay a benchmark 144 for an event, which is located in a text box142. Other methods for displaying an alert by email are possible.

The alert can also be received through a text message. FIG. 13 is ablock diagram 150 showing, by way of example, an alert 151 received viaa text message 152. The alert 151 is displayed on a cellular telephoneto notify the individual that an event has occurred. A benchmark 153 isalso displayed. Other displays and delivery methods for an alert arepossible.

Each type of alert can include response action elements for allowingrecipients of the alert to perform actions in response to an event. FIG.14 is a screen shot 160 showing, by way of example, an alert 161 withresponse action elements. The response action elements can includeaction buttons 166, text recommendations 163, hyperlinks 164, and images165. The alert is displayed in an email 162 and notifies a recipientthat a particular student has earned a perfect score on Activity 1.1.3in a precalculus class.

The text recommendations 163 can provide a process, list of recommendedsteps, detailed information of the event, historical information,subscription numbers and cost, or contact information to the recipient.The hyperlinks allow the recipient to quickly and easily access a Webpage for performing a response action. In addition to the hyperlinks,the response action buttons 166 can also allow the recipient to initiatea response action. The response action buttons 166 can include responseactions, such as forwarding a message, initiating a communication,recording an event, and purchasing additional subscriptions. Images,such as an award medal, copy of a grade book, and videos can also bedisplayed in the alert. Other types of response action elements andresponse actions are possible.

The recipient of an alert may be part of an education support groupcreated for an individual of the online education environment, such asthe student. The education support group includes multiple individualswith an interest in education-related information or in the particularstudent enrolled in the online learning environment. Alerts can betransmitted to one or more of the individuals based on a type of thealert. Further, the response actions provided by the response actionelements can be selected based on a type of the individual. For example,when a student receives a perfect score, an alert is sent to aneducation support group associated with the student. The educationsupport group may include a teacher, parent, and mentor, as well asother individuals with an interest in the student's educationalprogress. The alert received by the teacher may include response actionelements for response actions, such as sending a congratulations messageto the student and a hyperlink to a grade book for ensuring the perfectscore is recorded. Since the parent and mentor are unable to access thegrade book, their alerts contain only the response action element forsending the congratulations message.

In a further embodiment, the alerts can be customized by the onlinelearning staff, teachers, students, parents, or administrators. Thecustomization can include layout, text, hyperlinks, animation, color,and sound. Other types of customization are possible.

In a further embodiment, the education-related information can be usedto generate comparisons within, between, and among different groupingsof individuals and organizations associated with the online learningenvironment. The individuals can include administrators, teachers,students, and others. The comparisons can be based on a comparisonfactor, such as location, age, social economic class, dropout rate,organization size, ethnic background, student behavior, or thegroupings.

In addition, the comparisons can be used to evaluate the progress of oneor more individuals or organizations. For instance, a first teacherteaches a geometry class for a group of students located in the city ofSeattle, Wash. Upon termination of the course, cumulative scores foreach student are combined to generate an evaluation indicator for thegeometry class. The evaluation indicator can then be used for comparingwith another evaluation indicator for a geometry class taught by asecond teacher under the same conditions. The same conditions can beevaluated using the comparison factors. Thus, the second teacher mayteach a geometry class in the city of Portland, Oreg., which is fairlycomparable to Seattle in size, population, and ethnic background. Thecomparison of the indicators can be used to evaluate the teachers'performance, as well as the students' performance.

In a further embodiment, an event occurrence may have an effect on theoccurrence of an additional event. For example, a student has beencaught cheating. The process and rules for determining cheating ischanged to a lower standard for that student to identify furtherinstances of suspected cheating.

In yet a further embodiment, an individual can override a particularalert by selecting an override button or by entering a particularsubject matter of the alert. The override prevents individuals fromreceiving alerts they would ordinarily receive, such as during a meetingor online class seminar.

While the invention has been particularly shown and described asreferenced to the embodiments thereof, those skilled in the art willunderstand that the foregoing and other changes in form and detail maybe made therein without departing from the spirit and scope of theinvention.

What is claimed is:
 1. A computer-implemented system for providingperformance alerts in an online learning environment, comprising: adatabase to store educated-related information for a user within anonline learning environment; a server comprising a central processingunit, memory, an input port to receive the education-relatedinformation, and an output port, wherein the central processing unit isconfigured to: receive an assignment completed by the user; determine ascore of the assignment as below passing level; identify otherassignments that are the same as the received assignment and areassociated with below passing scores of the user; total a number of thereceived assignment and the same assignments as common assignments;apply a predetermined threshold to the number of common assignments;when the number of common assignments exceeds the predeterminedthreshold, block access of the user to the online learning environment;select a template for an alert based on the user's blocked access to theonline learning environment; and deliver the alert comprising notice ofthe user's blocked access.
 2. A system according to claim 1, wherein thealert is delivered to one or more of the user, an individual associatedwith the user, a teaching facilitator, teacher, or administrator.
 3. Asystem according to claim 1, wherein the central processing unitgenerates the alert, comprising: filling in the template; anddetermining recipients of the alert.
 4. A system according to claim 1,wherein the central processing unit delivers the alert via email,instant messaging, text messaging, or display on a website.
 5. A systemaccording to claim 1, wherein the central processing unit provides atleast one response action element with the alert, wherein each of theresponse action elements comprise an action button, recommendation,hyperlink, or image.
 6. A system according to claim 1, wherein thecentral processing unit forms a customized group of recipients for theuser and delivers the alert to the recipients in the customized group.7. A system according to claim 1, wherein the central processing unitreceives a request for performance of an action from a recipient of thealert and processes the request, comprising one or more of: generating aresponse alert; initiating a change to the education-related informationin the online learning environment; and reinstating access of the userto the online learning environment.
 8. A system according to claim 1,wherein the central processing unit reinstates access of the user to theonline learning environment based on one or more of instructions from ateaching facilitator, teacher, or administrator.
 9. A system accordingto claim 1, wherein the education alert is monitored on one of acontinuous basis and a periodic basis.
 10. A computer-implemented methodfor providing performance alerts in an online learning environment,comprising: monitoring education-related information for a user withinan online learning environment; receiving an assignment completed by theuser; determining a score of the assignment as below passing level;identifying other assignments that are the same as the receivedassignment and are associated with below passing scores of the user;totaling a number of the received assignment and the same assignments ascommon assignments; applying a predetermined threshold to the number ofcommon assignments; when the number of common assignments exceeds thepredetermined threshold, blocking access of the user to the onlinelearning environment; selecting a template for an alert based on theuser's blocked access to the online learning environment; and deliveringthe alert comprising notice of the user's blocked access.
 11. A methodaccording to claim 10, wherein the alert is delivered to one or more ofthe user, a family member of the user, a teaching facilitator, teacher,or administrator.
 12. A method according to claim 10, furthercomprising: generating the alert, comprising: filling in the template;and determining recipients of the alert.
 13. A method according to claim10, further comprising: delivering the alert via email, instantmessaging, text messaging, or display on a website.
 14. A methodaccording to claim 10, further comprising: providing at least oneresponse action element with the alert, wherein each of the responseaction elements comprise one of an action button, recommendation,hyperlink, and image.
 15. A method according to claim 10, furthercomprising: forming a customized group of recipients for the user; anddelivering the alert to the recipients in the customized group.
 16. Amethod according to claim 10, further comprising: receiving a requestfor performance of an action from a recipient of the alert; andprocessing the request, comprising one or more of: generating a responsealert; initiating a change to the education-related information in theonline learning environment; and reinstating access of the user to theonline learning environment.
 17. A method according to claim 10, furthercomprising: adding access of the user to the online learning environmentbased on one or more of instructions from a teaching facilitator,teacher, or administrator.
 18. A method according to claim 10, whereinthe education alert is monitored on one of a continuous basis and aperiodic basis.